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1  Introduction 


This  report  documents  research  performed  under  the  contract  entitled,  “Vision-Based  Control  of  Ag¬ 
ile,  Autonomous  Micro  Air  Vehicles  and  Small  UAVs  in  Urban  Environments.”  This  project  was  a 
5-year  effort  meant  to  mature  technologies  related  to  vision-based  feedback  to  enable  autonomous 
operation  within  urban  environments. 

The  technical  achievements  from  this  project  was  quite  substantial  in  that  many  contributions 
were  made  to  a  variety  of  disciplines.  In  particular,  substantial  progress  was  made  in  the  areas  of 
feature-point  tracking,  state  estimation,  scene  reconstruction,  robustness  to  camera  calibration,  daisy- 
chaining  navigation,  mapping,  path  planning,  and  feedback  characterization.  An  integrated  approach 
was  used  that  focused  on  multi-disciplinary  analysis  for  decision  making  and  control  commands. 

The  project  spiraled  the  maturation  of  technologies  from  theory  to  simulation  to  flight  testing.  The 
simulation  relied  upon  a  hardware-in-the-loop  simulation  (HILS)  facility  that  allowed  the  physical 
camera  to  record  measurements  from  high-fidelity  graphics  and  thus  consider  the  effects  of  distortion 
and  nonlinearities.  The  flight  testing  used  a  set  of  24-inch  wingspan  vehicles  carrying  a  commercial 
autopilot  while  a  laptop  analyzed  video  and  generated  guidance  commands  via  telemetry.  The  result¬ 
ing  setup  was  able  to  achieve  some  level  of  mission  capability  while  demonstrating  both  the  strengths, 
and  numerous  unsolved  challenges,  for  vision-based  control  of  aerial  robotics. 

Additionally,  the  project  has  further  developed  and  matured  the  relationahip  between  the  Air 
Force,  and  particularly  Eglin  Air  Force  Base,  with  the  research  team.  The  satellite  campus  known 
as  the  REEF  has  shown  value  as  a  hub  for  collaboration  between  academia  and  industry  to  the  Air 
Force.  The  combination  of  a  visualization  laboratory  and  micro  air  vehicle  facilities,  along  with  a 
technical  workforce,  is  a  formidable  pairing  that  will  continue  to  advance  mission  capabilities  of 
aerial  robotics  for  the  warfighter. 


2  Summary  of  Results 

2.1  Academic  Goals 

The  academic  goals  related  to  the  education  and  training  of  students.  As  such,  the  granting  of  degrees 
along  with  publishing  of  academic  papers  are  valuable  metrics  with  which  to  evaluate  the  academics. 
The  project  showed  particularly  strong  growth  in  this  area  due  to  the  extended  continuity  of  funding 
that  allowed  students  to  mature  while  working  on  the  same  research  project  throughout  their  studies. 

The  team  initially  hired  several  new  students  in  2003  who  remained  on  the  project  until  their 
graduation  with  a  PhD  degree  in  2007.  The  following  tables  relates  the  number  of  students  who  were 
primarly  funded  off  this  grant. 


2003 

2004 

2005 

2006 

2007 

M.S. 

3 

8 

7 

2 

2 

Ph.D. 

0 

1 

2 

1 

8 

The  publishing  of  papers  occured  in  both  conferences  and  journals.  A  strong  growth  is  again 
demonstrated  that  indicates  conference  papers  were  transitioned  into  journal  papers  during  the  final 
years  of  the  grant. 
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2003 

2004 

2005 

2006 

review 

preparation 

conference 

5 

28 

33 

26 

18 

20 

journal 

0 

3 

9 

4 

14 

8 

2.2  Technology  Transfer  Goals 

The  team  has  pursued  several  avenues  for  technology  transfer.  Certainly  the  transition  of  technologies 
to  the  warfighter  was  a  critical  concern  throughout  the  project.  These  various 

•  The  team  conducted  a  short  course  entitled  “Vision-Based  Control  for  Autonomous  Vehicles” 
held  at  the  AIAA  Guidance,  Navigation  and  Control  Conference  in  August  2007.  The  2-day 
course  had  12  attendees  from  4  countries. 

•  The  team  is  preparing  a  textbook  entitled  “Active  Vision  Control  for  Agile  Autonomous  Ve¬ 
hicles”  that  will  document  much  of  the  research  for  this  grant.  It  will  be  distributed  through 
AIAA  with  a  pagccount  sufficient  to  present  material  in  more  depth  than  can  be  obtained  in  a 
journal  paper. 

•  The  team  has  been  participating  on  a  number  of  SBIR  and  STTR  programs  in  the  areas  of 
aeroclasticity,  control,  design,  sensing,  and  dynamics. 

•  The  tema  has  applied  for  a  pair  of  patents  on  vehicle  design  and  sensor  analysis  that  directly 
result  from  this  grant. 

2.3  Programmatic  Goals 

The  programmatic  goals  related  primarily  to  the  technical  aspect  of  the  project.  The  team  investigated 
a  number  of  challenges  across  several  fields  in  a  multi-discplinary  fashion.  The  number  of  students 
and  papers  associated  with  this  grant  are  certainly  indicative  of  the  resulting  technical  contributions. 

•  high-fidelity  modeling  of  aeroelastic  MAV  wings 

•  data  fusion  of  IMU/vision  for  robust  vision 

•  multiresolution  learning  for  scene  reconstruction 

•  method  to  bound  uncertainty  in  camera  calibration 

•  nonlinear  stability  analysis  for  visual-servo  control 

•  3-D  motion  planning  for  mapping  using  sensor  quality 

A  number  of  lessons  were  also  learned  in  this  project.  These  lessons  learned  are  valuable  contri¬ 
butions  that  will  be  used  to  guide  future  projects.  In  particular,  the  understanding  of  limitations  in  the 
technology  and  associated  physical  reasoning  are  quite  useful. 

•  scale  of  aerodynamic/structure  complicated  for  MAV 

•  noise  is  devastating  for  vision-based  control 
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•  interlaced  cameras  worse  for  noise  than  progressive  cameras 

•  state  estimation  better  for  rotation  than  translation 

•  field  of  view  can  be  constrictive  for  llight  operations 

•  on-board  hardware  challenging  for  MAV 

2.4  Collaboration  Goals 

The  collaborative  goals  sought  to  foster  a  stronger  relationship  between  academia  and  the  Air  Force. 
The  team  enforced  compatibility,  and  often  equality,  in  both  hardware  and  software  as  a  foundation 
to  help  collaboration.  Also,  the  academics  advised  a  number  of  military  personnel  in  their  pursuit  of 
advanced  degrees. 

•  Phil  Webb  of  AFRL/MN 

•  Michael  Kaiser  of  AFRL/MN 

•  Crystal  Evans  of  Seek  Eagle 

•  Tony  Thomson  of  AFRL/MN 

•  Paul  Wilson  of  AFRL/MN 

•  David  Eaton  of  AFRL/MN 

•  Eric  Duron  of  Seek  Eagle 
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